Potential impact of annual vaccination with reformulated COVID-19 vaccines: Lessons from the US COVID-19 scenario modeling hub.

BACKGROUND: Coronavirus Disease 2019 (COVID-19) continues to cause significant hospitalizations and deaths in the United States. Its continued burden and the impact of annually reformulated vaccines remain unclear. Here, we present projections of COVID-19 hospitalizations and deaths in the United States for the next 2 years under 2 plausible assumptions about immune escape (20% per year and 50% per year) and 3 possible CDC recommendations for the use of annually reformulated vaccines (no recommendation, vaccination for those aged 65 years and over, vaccination for all eligible age groups based on FDA approval). METHODS AND FINDINGS: The COVID-19 Scenario Modeling Hub solicited projections of COVID-19 hospitalization and deaths between April 15, 2023 and April 15, 2025 under 6 scenarios representing the intersection of considered levels of immune escape and vaccination. Annually reformulated vaccines are assumed to be 65% effective against symptomatic infection with strains circulating on June 15 of each year and to become available on September 1. Age- and state-specific coverage in recommended groups was assumed to match that seen for the first (fall 2021) COVID-19 booster. State and national projections from 8 modeling teams were ensembled to produce projections for each scenario and expected reductions in disease outcomes due to vaccination over the projection period. From April 15, 2023 to April 15, 2025, COVID-19 is projected to cause annual epidemics peaking November to January. In the most pessimistic scenario (high immune escape, no vaccination recommendation), we project 2.1 million (90% projection interval (PI) [1,438,000, 4,270,000]) hospitalizations and 209,000 (90% PI [139,000, 461,000]) deaths, exceeding pre-pandemic mortality of influenza and pneumonia. In high immune escape scenarios, vaccination of those aged 65+ results in 230,000 (95% confidence interval (CI) [104,000, 355,000]) fewer hospitalizations and 33,000 (95% CI [12,000, 54,000]) fewer deaths, while vaccination of all eligible individuals results in 431,000 (95% CI: 264,000-598,000) fewer hospitalizations and 49,000 (95% CI [29,000, 69,000]) fewer deaths. CONCLUSIONS: COVID-19 is projected to be a significant public health threat over the coming 2 years. Broad vaccination has the potential to substantially reduce the burden of this disease, saving tens of thousands of lives each year.

COVSIM: A stochastic agent-based COVID-19 SIMulation model for North Carolina.

We document the evolution and use of the stochastic agent-based COVID-19 simulation model (COVSIM) to study the impact of population behaviors and public health policy on disease spread within age, race/ethnicity, and urbanicity subpopulations in North Carolina. We detail the methodologies used to model the complexities of COVID-19, including multiple agent attributes (i.e., age, race/ethnicity, high-risk medical status), census tract-level interaction network, disease state network, agent behavior (i.e., masking, pharmaceutical intervention (PI) uptake, quarantine, mobility), and variants. We describe its uses outside of the COVID-19 Scenario Modeling Hub (CSMH), which has focused on the interplay of nonpharmaceutical and pharmaceutical interventions, equitability of vaccine distribution, and supporting local county decision-makers in North Carolina. This work has led to multiple publications and meetings with a variety of local stakeholders. When COVSIM joined the CSMH in January 2022, we found it was a sustainable way to support new COVID-19 challenges and allowed the group to focus on broader scientific questions. The CSMH has informed adaptions to our modeling approach, including redesigning our high-performance computing implementation.

Evaluating the potential impact of rubella-containing vaccine introduction on congenital rubella syndrome in Afghanistan, Dem. Republic of Congo, Ethiopia, Nigeria, and Pakistan: A mathematical modeling study.

We assessed the potential impact of introducing rubella-containing vaccine (RCV) on congenital rubella syndrome (CRS) incidence in Afghanistan (AFG), Democratic Republic of Congo (COD), Ethiopia (ETH), Nigeria (NGA), and Pakistan (PAK). We simulated several RCV introduction scenarios over 30 years using a validated mathematical model. Our findings indicate that RCV introduction could avert between 86,000 and 535,000 CRS births, preventing 2.5 to 15.8 million disability-adjusted life years. AFG and PAK could reduce about 90% of CRS births by introducing RCV with current measles routine coverage and executing supplemental immunization activities (SIAs). However, COD, NGA, and ETH must increase their current routine vaccination coverage to reduce CRS incidence significantly. This study showcases the potential benefits of RCV introduction and reinforces the need for global action to strengthen immunization programs.

Potential impact of annual vaccination with reformulated COVID-19 vaccines: lessons from the U.S. COVID-19 Scenario Modeling Hub.

Importance: COVID-19 continues to cause significant hospitalizations and deaths in the United States. Its continued burden and the impact of annually reformulated vaccines remain unclear. Objective: To project COVID-19 hospitalizations and deaths from April 2023-April 2025 under two plausible assumptions about immune escape (20% per year and 50% per year) and three possible CDC recommendations for the use of annually reformulated vaccines (no vaccine recommendation, vaccination for those aged 65+, vaccination for all eligible groups). Design: The COVID-19 Scenario Modeling Hub solicited projections of COVID-19 hospitalization and deaths between April 15, 2023-April 15, 2025 under six scenarios representing the intersection of considered levels of immune escape and vaccination. State and national projections from eight modeling teams were ensembled to produce projections for each scenario. Setting: The entire United States. Participants: None. Exposure: Annually reformulated vaccines assumed to be 65% effective against strains circulating on June 15 of each year and to become available on September 1. Age and state specific coverage in recommended groups was assumed to match that seen for the first (fall 2021) COVID-19 booster. Main outcomes and measures: Ensemble estimates of weekly and cumulative COVID-19 hospitalizations and deaths. Expected relative and absolute reductions in hospitalizations and deaths due to vaccination over the projection period. Results: From April 15, 2023-April 15, 2025, COVID-19 is projected to cause annual epidemics peaking November-January. In the most pessimistic scenario (high immune escape, no vaccination recommendation), we project 2.1 million (90% PI: 1,438,000-4,270,000) hospitalizations and 209,000 (90% PI: 139,000-461,000) deaths, exceeding pre-pandemic mortality of influenza and pneumonia. In high immune escape scenarios, vaccination of those aged 65+ results in 230,000 (95% CI: 104,000-355,000) fewer hospitalizations and 33,000 (95% CI: 12,000-54,000) fewer deaths, while vaccination of all eligible individuals results in 431,000 (95% CI: 264,000-598,000) fewer hospitalizations and 49,000 (95% CI: 29,000-69,000) fewer deaths. Conclusion and Relevance: COVID-19 is projected to be a significant public health threat over the coming two years. Broad vaccination has the potential to substantially reduce the burden of this disease.

Evaluation of the US COVID-19 Scenario Modeling Hub for informing pandemic response under uncertainty.

Our ability to forecast epidemics far into the future is constrained by the many complexities of disease systems. Realistic longer-term projections may, however, be possible under well-defined scenarios that specify the future state of critical epidemic drivers. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make months ahead projections of SARS-CoV-2 burden, totaling nearly 1.8 million national and state-level projections. Here, we find SMH performance varied widely as a function of both scenario validity and model calibration. We show scenarios remained close to reality for 22 weeks on average before the arrival of unanticipated SARS-CoV-2 variants invalidated key assumptions. An ensemble of participating models that preserved variation between models (using the linear opinion pool method) was consistently more reliable than any single model in periods of valid scenario assumptions, while projection interval coverage was near target levels. SMH projections were used to guide pandemic response, illustrating the value of collaborative hubs for longer-term scenario projections.

Quantifying association and disparities between diabetes complications and COVID-19 outcomes: A retrospective study using electronic health records.

BACKGROUND: Despite established relationships between diabetic status and an increased risk for COVID-19 severe outcomes, there is a limited number of studies examining the relationships between diabetes complications and COVID-19-related risks. We use the Adapted Diabetes Complications Severity Index to define seven diabetes complications. We aim to understand the risk for COVID-19 infection, hospitalization, mortality, and longer length of stay of diabetes patients with complications. METHODS: We perform a retrospective case-control study using Electronic Health Records (EHRs) to measure differences in the risks for COVID-19 severe outcomes amongst those with diabetes complications. Using multiple logistic regression, we calculate adjusted odds ratios (OR) for COVID-19 infection, hospitalization, and in-hospital mortality of the case group (patients with diabetes complications) compared to a control group (patients without diabetes). We also calculate adjusted mean difference in length of stay between the case and control groups using multiple linear regression. RESULTS: Adjusting demographics and comorbidities, diabetes patients with renal complications have the highest odds for COVID-19 infection (OR = 1.85, 95% CI = [1.71, 1.99]) while those with metabolic complications have the highest odds for COVID-19 hospitalization (OR = 5.58, 95% CI = [3.54, 8.77]) and in-hospital mortality (OR = 2.41, 95% CI = [1.35, 4.31]). The adjusted mean difference (MD) of hospital length-of-stay for diabetes patients, especially those with cardiovascular (MD = 0.94, 95% CI = [0.17, 1.71]) or peripheral vascular (MD = 1.72, 95% CI = [0.84, 2.60]) complications, is significantly higher than non-diabetes patients. African American patients have higher odds for COVID-19 infection (OR = 1.79, 95% CI = [1.66, 1.92]) and hospitalization (OR = 1.62, 95% CI = [1.39, 1.90]) than White patients in the general diabetes population. However, White diabetes patients have higher odds for COVID-19 in-hospital mortality. Hispanic patients have higher odds for COVID-19 infection (OR = 2.86, 95% CI = [2.42, 3.38]) and shorter mean length of hospital stay than non-Hispanic patients in the general diabetes population. Although there is no significant difference in the odds for COVID-19 hospitalization and in-hospital mortality between Hispanic and non-Hispanic patients in the general diabetes population, Hispanic patients have higher odds for COVID-19 hospitalization (OR = 1.83, 95% CI = [1.16, 2.89]) and in-hospital mortality (OR = 3.69, 95% CI = [1.18, 11.50]) in the diabetes population with no complications. CONCLUSIONS: The presence of diabetes complications increases the risks of COVID-19 infection, hospitalization, and worse health outcomes with respect to in-hospital mortality and longer hospital length of stay. We show the presence of health disparities in COVID-19 outcomes across demographic groups in our diabetes population. One such disparity is that African American and Hispanic diabetes patients have higher odds of COVID-19 infection than White and Non-Hispanic diabetes patients, respectively. Furthermore, Hispanic patients might have less access to the hospital care compared to non-Hispanic patients when longer hospitalizations are needed due to their diabetes complications. Finally, diabetes complications, which are generally associated with worse COVID-19 outcomes, might be predominantly determining the COVID-19 severity in those infected patients resulting in less demographic differences in COVID-19 hospitalization and in-hospital mortality.

Informing pandemic response in the face of uncertainty. An evaluation of the U.S. COVID-19 Scenario Modeling Hub.

Our ability to forecast epidemics more than a few weeks into the future is constrained by the complexity of disease systems, our limited ability to measure the current state of an epidemic, and uncertainties in how human action will affect transmission. Realistic longer-term projections (spanning more than a few weeks) may, however, be possible under defined scenarios that specify the future state of critical epidemic drivers, with the additional benefit that such scenarios can be used to anticipate the comparative effect of control measures. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make 6-month ahead projections of the number of SARS-CoV-2 cases, hospitalizations and deaths. The SMH released nearly 1.8 million national and state-level projections between February 2021 and November 2022. SMH performance varied widely as a function of both scenario validity and model calibration. Scenario assumptions were periodically invalidated by the arrival of unanticipated SARS-CoV-2 variants, but SMH still provided projections on average 22 weeks before changes in assumptions (such as virus transmissibility) invalidated scenarios and their corresponding projections. During these periods, before emergence of a novel variant, a linear opinion pool ensemble of contributed models was consistently more reliable than any single model, and projection interval coverage was near target levels for the most plausible scenarios (e.g., 79% coverage for 95% projection interval). SMH projections were used operationally to guide planning and policy at different stages of the pandemic, illustrating the value of the hub approach for long-term scenario projections.

Engaging stakeholders in the use of an interactive simulation tool to support decision-making about the implementation of colorectal cancer screening interventions.

PURPOSE: We aimed to understand how an interactive, web-based simulation tool can be optimized to support decision-making about the implementation of evidence-based interventions (EBIs) for improving colorectal cancer (CRC) screening. METHODS: Interviews were conducted with decision-makers, including health administrators, advocates, and researchers, with a strong foundation in CRC prevention. Following a demonstration of the microsimulation modeling tool, participants reflected on the tool's potential impact for informing the selection and implementation of strategies for improving CRC screening and outcomes. The interviews assessed participants' preferences regarding the tool's design and content, comprehension of the model results, and recommendations for improving the tool. RESULTS: Seventeen decision-makers completed interviews. Themes regarding the tool's utility included building a case for EBI implementation, selecting EBIs to adopt, setting implementation goals, and understanding the evidence base. Reported barriers to guiding EBI implementation included the tool being too research-focused, contextual differences between the simulated and local contexts, and lack of specificity regarding the design of simulated EBIs. Recommendations to address these challenges included making the data more actionable, allowing users to enter their own model inputs, and providing a how-to guide for implementing the simulated EBIs. CONCLUSION: Diverse decision-makers found the simulation tool to be most useful for supporting early implementation phases, especially deciding which EBI(s) to implement. To increase the tool's utility, providing detailed guidance on how to implement the selected EBIs, and the extent to which users can expect similar CRC screening gains in their contexts, should be prioritized.

Modeling the Impact of Nonpharmaceutical Interventions on COVID-19 Transmission in K-12 Schools.

Background. The novel coronavirus SARS-CoV-2 spread across the world causing many waves of COVID-19. Children were at high risk of being exposed to the disease because they were not eligible for vaccination during the first 20 mo of the pandemic in the United States. While children 5 y and older are now eligible to receive a COVID-19 vaccine in the United States, vaccination rates remain low despite most schools returning to in-person instruction. Nonpharmaceutical interventions (NPIs) are important for controlling the spread of COVID-19 in K-12 schools. US school districts used varied and layered mitigation strategies during the pandemic. The goal of this article is to analyze the impact of different NPIs on COVID-19 transmission within K-12 schools. Methods. We developed a deterministic stratified SEIR model that captures the role of social contacts between cohorts in disease transmission to estimate COVID-19 incidence under different NPIs including masks, random screening, contact reduction, school closures, and test-to-stay. We designed contact matrices to simulate the contact patterns between students and teachers within schools. We estimated the proportion of susceptible infected associated with each intervention over 1 semester under the Omicron variant. Results. We find that masks and reducing contacts can greatly reduce new infections among students. Weekly screening tests also have a positive impact on disease mitigation. While self-quarantining symptomatic infections and school closures are effective measures for decreasing semester-end infections, they increase absenteeism. Conclusion. The model provides a useful tool for evaluating the impact of a variety of NPIs on disease transmission in K-12 schools. While the model is tested under Omicron variant parameters in US K-12 schools, it can be adapted to study other populations under different disease settings. Highlights: A stratified SEIR model was developed that captures the role of social contacts in K-12 schools to estimate COVID-19 transmission under different nonpharmaceutical interventions.While masks, random screening, contact reduction, school closures, and test-to-stay are all beneficial interventions, masks and contact reduction resulted in the greatest reduction in new infections among students from the tested scenarios.Layered interventions provide more benefits than implementing interventions independently.

Cross-sector decision landscape in response to COVID-19: A qualitative network mapping analysis of North Carolina decision-makers.

Introduction: The COVID-19 pandemic response has demonstrated the interconnectedness of individuals, organizations, and other entities jointly contributing to the production of community health. This response has involved stakeholders from numerous sectors who have been faced with new decisions, objectives, and constraints. We examined the cross-sector organizational decision landscape that formed in response to the COVID-19 pandemic in North Carolina. Methods: We conducted virtual semi-structured interviews with 44 organizational decision-makers representing nine sectors in North Carolina between October 2020 and January 2021 to understand the decision-making landscape within the first year of the COVID-19 pandemic. In line with a complexity/systems thinking lens, we defined the decision landscape as including decision-maker roles, key decisions, and interrelationships involved in producing community health. We used network mapping and conventional content analysis to analyze transcribed interviews, identifying relationships between stakeholders and synthesizing key themes. Results: Decision-maker roles were characterized by underlying tensions between balancing organizational mission with employee/community health and navigating organizational vs. individual responsibility for reducing transmission. Decision-makers' roles informed their perspectives and goals, which influenced decision outcomes. Key decisions fell into several broad categories, including how to translate public health guidance into practice; when to institute, and subsequently loosen, public health restrictions; and how to address downstream social and economic impacts of public health restrictions. Lastly, given limited and changing information, as well as limited resources and expertise, the COVID-19 response required cross-sector collaboration, which was commonly coordinated by local health departments who had the most connections of all organization types in the resulting network map. Conclusions: By documenting the local, cross-sector decision landscape that formed in response to COVID-19, we illuminate the impacts different organizations may have on information/misinformation, prevention behaviors, and, ultimately, health. Public health researchers and practitioners must understand, and work within, this complex decision landscape when responding to COVID-19 and future community health challenges.

Assessing the impact of multicomponent interventions on colorectal cancer screening through simulation: What would it take to reach national screening targets in North Carolina?

Healthy People 2020 and the National Colorectal Cancer Roundtable established colorectal cancer (CRC) screening targets of 70.5% and 80%, respectively. While evidence-based interventions (EBIs) have increased CRC screening, the ability to achieve these targets at the population level remains uncertain. We simulated the impact of multicomponent interventions in North Carolina over 5 years to assess the potential for meeting national screening targets. Each intervention scenario is described as a core EBI with additional components indicated by the "+" symbol: patient navigation for screening colonoscopy (PN-for-Col+), mailed fecal immunochemical testing (MailedFIT+), MailedFIT+ targeted to Medicaid enrollees (MailedFIT + forMd), and provider assessment and feedback (PAF+). Each intervention was simulated with and without Medicaid expansion and at different levels of exposure (i.e., reach) for targeted populations. Outcomes included the percent up-to-date overall and by sociodemographic subgroups and number of CRC cases and deaths averted. Each multicomponent intervention was associated with increased CRC screening and averted both CRC cases and deaths; three had the potential to reach screening targets. PN-for-Col + achieved the 70.5% target with 97% reach after 1 year, and the 80% target with 78% reach after 5 years. MailedFIT+ achieved the 70.5% target with 74% reach after 1 year and 5 years. In the Medicaid population, assuming Medicaid expansion, MailedFIT + forMd reached the 70.5% target after 5 years with 97% reach. This study clarifies the potential for states to reach national CRC screening targets using multicomponent EBIs, but decision-makers also should consider tradeoffs in cost, reach, and ability to reduce disparities when selecting interventions.

Vaccinating children against COVID-19 is crucial to protect schools and communities.

To evaluate the joint impact of childhood vaccination rates and school masking policies on community transmission and severe outcomes due to COVID-19, we utilized a stochastic, agent-based simulation of North Carolina to test 24 health policy scenarios. In these scenarios, we varied the childhood (ages 5 to 19) vaccination rate relative to the adult's (ages 20 to 64) vaccination rate and the masking relaxation policies in schools. We measured the overall incidence of disease, COVID-19-related hospitalization, and mortality from 2021 July 1 to 2023 July 1. Our simulation estimates that removing all masks in schools in January 2022 could lead to a 31% to 45%, 23% to 35%, and 13% to 19% increase in cumulative infections for ages 5 to 9, 10 to 19, and the total population, respectively, depending on the childhood vaccination rate. Additionally, achieving a childhood vaccine uptake rate of 50% of adults could lead to a 31% to 39% reduction in peak hospitalizations overall masking scenarios compared with not vaccinating this group. Finally, our simulation estimates that increasing vaccination uptake for the entire eligible population can reduce peak hospitalizations in 2022 by an average of 83% and 87% across all masking scenarios compared to the scenarios where no children are vaccinated. Our simulation suggests that high vaccination uptake among both children and adults is necessary to mitigate the increase in infections from mask removal in schools and workplaces.

When History and Heterogeneity Matter: A Tutorial on the Impact of Markov Model Specifications in the Context of Colorectal Cancer Screening.

BACKGROUND: Markov models are used in health research to simulate health care utilization and disease states over time. Health phenomena, however, are complex, and the memoryless assumption of Markov models may not appropriately represent reality. This tutorial provides guidance on the use of Markov models of different orders and stratification levels in health decision-analytic modeling. Colorectal cancer (CRC) screening is used as a case example to examine the impact of using different Markov modeling approaches on CRC outcomes. METHODS: This study used insurance claims data from commercially insured individuals in Oregon to estimate transition probabilities between CRC screening states (no screen, colonoscopy, fecal immunochemical test or fecal occult blood test). First-order, first-order stratified by sex and geography, and third-order Markov models were compared. Screening trajectories produced from the different Markov models were incorporated into a microsimulation model that simulated the natural history of CRC disease progression. Simulation outcomes (e.g., future screening choices, CRC incidence, deaths due to CRC) were compared across models. RESULTS: Simulated CRC screening trajectories and resulting CRC outcomes varied depending on the Markov modeling approach used. For example, when using the first-order, first-order stratified, and third-order Markov models, 30%, 31%, and 44% of individuals used colonoscopy as their only screening modality, respectively. Screening trajectories based on the third-order Markov model predicted that a higher percentage of individuals were up-to-date with CRC screening as compared with the other Markov models. LIMITATIONS: The study was limited to insurance claims data spanning 5 y. It was not possible to validate which Markov model better predicts long-term screening behavior and outcomes. CONCLUSIONS: Findings demonstrate the impact that different order and stratification assumptions can have in decision-analytic models. HIGHLIGHTS: This tutorial uses colorectal cancer screening as a case example to provide guidance on the use of Markov models of different orders and stratification levels in health decision-analytic models.Colorectal cancer screening trajectories and projected health outcomes were sensitive to the use of alternate Markov model specifications.Although data limitations precluded the assessment of model accuracy beyond a 5-y period, within the 5-y period, the third-order Markov model was slightly more accurate in predicting the fifth colorectal cancer screening action than the first-order Markov model.Findings from this tutorial demonstrate the importance of examining the memoryless assumption of the first-order Markov model when simulating health care utilization over time.

Can vaccine prioritization reduce disparities in COVID-19 burden for historically marginalized populations?

SARS-CoV-2 vaccination strategies were designed to reduce COVID-19 mortality, morbidity, and health inequities. To assess the impact of vaccination strategies on disparities in COVID-19 burden among historically marginalized populations (HMPs), e.g. Black race and Hispanic ethnicity, we used an agent-based simulation model, populated with census-tract data from North Carolina. We projected COVID-19 deaths, hospitalizations, and cases from 2020 July 1 to 2021 December 31, and estimated racial/ethnic disparities in COVID-19 outcomes. We modeled 2-stage vaccination prioritization scenarios applied to sub-groups including essential workers, older adults (65+), adults with high-risk health conditions, HMPs, or people in low-income tracts. Additionally, we estimated the effects of maximal uptake (100% for HMP vs. 100% for everyone), and distribution to only susceptible people. We found strategies prioritizing essential workers, then older adults led to the largest mortality and case reductions compared to no prioritization. Under baseline uptake scenarios, the age-adjusted mortality for HMPs was higher (e.g. 33.3%-34.1% higher for the Black population and 13.3%-17.0% for the Hispanic population) compared to the White population. The burden on HMPs decreased only when uptake was increased to 100% in HMPs; however, the Black population still had the highest relative mortality rate even when targeted distribution strategies were employed. If prioritization schemes were not paired with increased uptake in HMPs, disparities did not improve. The vaccination strategies publicly outlined were insufficient, exacerbating disparities between racial and ethnic groups. Strategies targeted to increase vaccine uptake among HMPs are needed to ensure equitable distribution and minimize disparities in outcomes.

The SMART Framework: Selection of Machine Learning Algorithms With ReplicaTions-A Case Study on the Microvascular Complications of Diabetes.

Over 34 million people in the US have diabetes, a major cause of blindness, renal failure, and amputations. Machine learning (ML) models can predict high-risk patients to help prevent adverse outcomes. Selecting the 'best' prediction model for a given disease, population, and clinical application is challenging due to the hundreds of health-related ML models in the literature and the increasing availability of ML methodologies. To support this decision process, we developed the Selection of Machine-learning Algorithms with ReplicaTions (SMART) Framework that integrates building and selecting ML models with decision theory. We build ML models and estimate performance for multiple plausible future populations with a replicated nested cross-validation technique. We rank ML models by simulating decision-maker priorities, using a range of accuracy measures (e.g., AUC) and robustness metrics from decision theory (e.g., minimax Regret). We present the SMART Framework through a case study on the microvascular complications of diabetes using data from the ACCORD clinical trial. We compare selections made by risk-averse, -neutral, and -seeking decision-makers, finding agreement in 80% of the risk-averse and risk-neutral selections, with the risk-averse selections showing consistency for a given complication. We also found that the models that best predicted outcomes in the validation set were those with low performance variance on the testing set, indicating a risk-averse approach in model selection is ideal when there is a potential for high population feature variability. The SMART Framework is a powerful, interactive tool that incorporates various ML algorithms and stakeholder preferences, generalizable to new data and technological advancements.

Association of Simulated COVID-19 Vaccination and Nonpharmaceutical Interventions With Infections, Hospitalizations, and Mortality.

Importance: Vaccination against SARS-CoV-2 has the potential to significantly reduce transmission and COVID-19 morbidity and mortality. The relative importance of vaccination strategies and nonpharmaceutical interventions (NPIs) is not well understood. Objective: To assess the association of simulated COVID-19 vaccine efficacy and coverage scenarios with and without NPIs with infections, hospitalizations, and deaths. Design, Setting, and Participants: An established agent-based decision analytical model was used to simulate COVID-19 transmission and progression from March 24, 2020, to September 23, 2021. The model simulated COVID-19 spread in North Carolina, a US state of 10.5 million people. A network of 1 017 720 agents was constructed from US Census data to represent the statewide population. Exposures: Scenarios of vaccine efficacy (50% and 90%), vaccine coverage (25%, 50%, and 75% at the end of a 6-month distribution period), and NPIs (reduced mobility, school closings, and use of face masks) maintained and removed during vaccine distribution. Main Outcomes and Measures: Risks of infection from the start of vaccine distribution and risk differences comparing scenarios. Outcome means and SDs were calculated across replications. Results: In the worst-case vaccination scenario (50% efficacy, 25% coverage), a mean (SD) of 2 231 134 (117 867) new infections occurred after vaccination began with NPIs removed, and a mean (SD) of 799 949 (60 279) new infections occurred with NPIs maintained during 11 months. In contrast, in the best-case scenario (90% efficacy, 75% coverage), a mean (SD) of 527 409 (40 637) new infections occurred with NPIs removed and a mean (SD) of 450 575 (32 716) new infections occurred with NPIs maintained. With NPIs removed, lower efficacy (50%) and higher coverage (75%) reduced infection risk by a greater magnitude than higher efficacy (90%) and lower coverage (25%) compared with the worst-case scenario (mean [SD] absolute risk reduction, 13% [1%] and 8% [1%], respectively). Conclusions and Relevance: Simulation outcomes suggest that removing NPIs while vaccines are distributed may result in substantial increases in infections, hospitalizations, and deaths. Furthermore, as NPIs are removed, higher vaccination coverage with less efficacious vaccines can contribute to a larger reduction in risk of SARS-CoV-2 infection compared with more efficacious vaccines at lower coverage. These findings highlight the need for well-resourced and coordinated efforts to achieve high vaccine coverage and continued adherence to NPIs before many prepandemic activities can be resumed.

High-Quality Masks Reduce COVID-19 Infections and Deaths in the US.

Objectives: To evaluate the effectiveness of widespread adoption of masks or face coverings to reduce community transmission of the SARS-CoV-2 virus that causes COVID-19. Methods: We created an agent-based stochastic network simulation using a variant of the standard SEIR dynamic infectious disease model. We considered a mask order that was initiated 3.5 months after the first confirmed COVID-19 case. We varied the likelihood of individuals wearing masks from 0-100% in steps of 20% (mask adherence) and considered 25% to 90% mask-related reduction in viral transmission (mask efficacy). Sensitivity analyses assessed early (by week 13) versus late (by week 42) adoption of masks and geographic differences in adherence (highest in urban and lowest in rural areas). Results: Introduction of mask use with 50% efficacy worn by 50% of individuals reduces the cumulative infection attack rate (IAR) by 27%, the peak prevalence by 49%, and population-wide mortality by 29%. If 90% of individuals wear 50% efficacious masks, this decreases IAR by 54%, peak prevalence by 75%, and population-wide mortality by 55%; similar improvements hold if 70% of individuals wear 75% efficacious masks. Late adoption reduces IAR and deaths by 18% or more compared to no adoption. Lower adoption in rural areas than urban would lead to rural areas having the highest IAR. Conclusions: Even after community transmission of SARS-CoV-2 has been established, adoption of mask-wearing by a majority of community-dwelling individuals can meaningfully reduce the number and outcome of COVID-19 infections over and above physical distancing interventions.

The Joint Impact of COVID-19 Vaccination and Non-Pharmaceutical Interventions on Infections, Hospitalizations, and Mortality: An Agent-Based Simulation.

Background: Vaccination against SARS-CoV-2 has the potential to significantly reduce transmission and morbidity and mortality due to COVID-19. This modeling study simulated the comparative and joint impact of COVID-19 vaccine efficacy and coverage with and without non-pharmaceutical interventions (NPIs) on total infections, hospitalizations, and deaths. Methods: An agent-based simulation model was employed to estimate incident SARS-CoV-2 infections and COVID-19-associated hospitalizations and deaths over 18 months for the State of North Carolina, a population of roughly 10.5 million. Vaccine efficacy of 50% and 90% and vaccine coverage of 25%, 50%, and 75% (at the end of a 6-month distribution period) were evaluated. Six vaccination scenarios were simulated with NPIs (i.e., reduced mobility, school closings, face mask usage) maintained and removed during the period of vaccine distribution. Results: In the worst-case vaccination scenario (50% efficacy and 25% coverage), 2,231,134 new SARS-CoV-2 infections occurred with NPIs removed and 799,949 infections with NPIs maintained. In contrast, in the best-case scenario (90% efficacy and 75% coverage), there were 450,575 new infections with NPIs maintained and 527,409 with NPIs removed. When NPIs were removed, lower efficacy (50%) and higher coverage (75%) reduced infection risk by a greater magnitude than higher efficacy (90%) and lower coverage (25%) compared to the worst-case scenario (absolute risk reduction 13% and 8%, respectively). Conclusion: Simulation results suggest that premature lifting of NPIs while vaccines are distributed may result in substantial increases in infections, hospitalizations, and deaths. Furthermore, as NPIs are removed, higher vaccination coverage with less efficacious vaccines can contribute to a larger reduction in risk of SARS-CoV-2 infection compared to more efficacious vaccines at lower coverage. Our findings highlight the need for well-resourced and coordinated efforts to achieve high vaccine coverage and continued adherence to NPIs before many pre-pandemic activities can be resumed.

Workflow Differences Affect Data Accuracy in Oncologic EHRs: A First Step Toward Detangling the Diagnosis Data Babel.

PURPOSE: Diagnosis (DX) information is key to clinical data reuse, yet accessible structured DX data often lack accuracy. Previous research hints at workflow differences in cancer DX entry, but their link to clinical data quality is unclear. We hypothesized that there is a statistically significant relationship between workflow-describing variables and DX data quality. METHODS: We extracted DX data from encounter and order tables within our electronic health records (EHRs) for a cohort of patients with confirmed brain neoplasms. We built and optimized logistic regressions to predict the odds of fully accurate (ie, correct neoplasm type and anatomic site), inaccurate, and suboptimal (ie, vague) DX entry across clinical workflows. We selected our variables based on correlation strength of each outcome variable. RESULTS: Both workflow and personnel variables were predictive of DX data quality. For example, a DX entered in departments other than oncology had up to 2.89 times higher odds of being accurate (P < .0001) compared with an oncology department; an outpatient care location had up to 98% fewer odds of being inaccurate (P < .0001), but had 458 times higher odds of being suboptimal (P < .0001) compared with main campus, including the cancer center; and a DX recoded by a physician assistant had 85% fewer odds of being suboptimal (P = .005) compared with those entered by physicians. CONCLUSION: These results suggest that differences across clinical workflows and the clinical personnel producing EHR data affect clinical data quality. They also suggest that the need for specific structured DX data recording varies across clinical workflows and may be dependent on clinical information needs. Clinicians and researchers reusing oncologic data should consider such heterogeneity when conducting secondary analyses of EHR data.

Notice to comply: A systematic review of clinician compliance with guidelines surrounding acute hospital-based infection management.

PURPOSE: To identify and characterize studies evaluating clinician compliance with infection-related guidelines, and to explore trends in guideline design and implementation strategies. DATA SOURCES: PubMed database, April 2017. Followed the PRISMA Statement for systematic reviews. STUDY SELECTION: Scope was limited to studies reporting compliance with guidelines pertaining to the prevention, detection, and/or treatment of acute hospital-based infections. Initial search (1,499 titles) was reduced to 49 selected articles. DATA EXTRACTION: Extracted publication and guideline characteristics, outcome measures reported, and any results related to clinician compliance. Primary summary measures were frequencies and distributions of characteristics. Interventions that led to improved compliance results were analyzed to identify trends in guideline design and implementation. RESULTS OF DATA SYNTHESIS: Of the 49 selected studies, 18 (37%), 13 (27%), and 10 (20%) focused on sepsis, pneumonia, and general infection, respectively. Six (12%), 17 (35%), and 26 (53%) studies assessed local, national, and international guidelines, respectively. Twenty studies (41%) reported 1-instance compliance results, 28 studies (57%) reported 2-instance compliance results (either before-and-after studies or control group studies), and 1 study (2%) described compliance qualitatively. Average absolute change in compliance for minimal, decision support, and multimodal interventions was 10%, 14%, and 25%, respectively. Twelve studies (24%) reported no patient outcome alongside compliance. CONCLUSIONS: Multimodal interventions and quality improvement initiatives seem to produce the greatest improvement in compliance, but trends in other factors were inconsistent. Additional research is required to investigate these relationships and understand the implications behind various approaches to guideline design, communication, and implementation, in addition to effectiveness of protocol impact on relevant patient outcomes.